Systems, methods, and user interfaces in a patent management system

ABSTRACT

A method may include receiving a search query from a computing device, the search query identifying an examiner in a patent office; in response to the receiving, gathering information associated with the examiner from at least one database, the information including experience metrics for the examiner and allowance metrics for the examiner; generating a user interface for display on the computing device; the user interface including: an experience section including a technology experience score for the examiner based on the experience metrics; and an allowance rate section that includes: allowance visualization options; and an allowance visualization based on the allowance metrics and a selected visualization option of the allowance visualization options.

CLAIM OF PRIORITY

This application is a continuation of and claims the benefit of priorityunder 35 U.S.C. § 120 to U.S. Patent Application Serial No. 15/183,093,filed on Jun. 15, 2016, which claims the benefit of priority under 35U.S.C. § 119(e) to U.S. Provisional Patent Application Ser. No.62/175,903, filed on Jun. 15, 2015, the benefit of priority of each ofwhich is claimed hereby, and which is incorporated by reference hereinin its entirety.

TECHNICAL FIELD

Embodiments described herein generally relate to user interfaces and inparticular, but without limitation, user interfaces in a patentmanagement system.

BACKGROUND

In order to obtain a patent an applicant submits a patent application toone or more patent offices. Then, an examiner conducts a search todetermine if the patent should be allowed.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings, which are not necessarily drawn to scale, like numeralsmay describe similar components in different views. Like numerals havingdifferent letter suffixes may represent different instances of similarcomponents. Some embodiments are illustrated by way of example, and notlimitation, in the figures of the accompanying drawings in which:

FIG. 1 is a schematic diagram of components of a patent analyticssystem, according to various examples;

FIG. 2 is an examiner overview interface, according to various examples;

FIG. 3 . is an abandoned cases notifier interface, according to variousexamples;

FIG. 4 is an expired cases notifier interface, according to variousexamples; and

FIG. 5 is a block diagram illustrating an example machine upon which anyone or more of the techniques (e.g., methodologies) discussed herein maybe performed, according to an example embodiment.

DETAILED DESCRIPTION

In various examples, an application provides data to inform patentees orother users about the performance of examiners, art units, and techcenters at a patent office such as the United States Patent andTrademark Office (USPTO). The data may be used to determine a course ofaction to take during examination of a patent or other researchpurposes. The application may also generate notices and predictions ofabandonments or expirations of applications or issued patents.

FIG. 1 is a schematic diagram of components of a patent analyticssystem, according to various examples. The patent analytics system 100may include a data analyzer module 102, patent database 104, web server106, examiner database 108, and notification module 110. The patentanalytics system 100 may transmit and receive data from experiencesources 112 and patent data sources 114 via a network such as theInternet. The patent analytics system 100 may also receive and transmitdata via the network to a computing device 116. While components 102-110are illustrated in a single block, the components may be located ondifferent computing devices and in different geographic locations.

In various examples, a user may use the computing device 116 (e.g.,desktop computer, laptop, tablet, mobile phone) to begin execution of anapplication to interact with or help generate the data described herein.The application may be stored on the computing device 116 or may beserved to the computing device 116 from a server, such as the web server106. In various examples, the data presented to the user in theapplication may be locally stored, remotely stored, dynamicallycalculated, or combinations thereof. While a single application isdescribed herein, multiple applications may be used. For example, oneapplication may be used to retrieve a profile of an examiner—asdescribed in more detail below—and a different application may be usedto monitor potential abandonments. For illustration purposes, theapplication will be discussed as a web application served from the webserver 106 to the computing device 116.

In an example, the patent database 104 is maintained on one or morestorage devices (not shown). The storage device(s) may be located in thesame computing device—such as patent analytics system 100—or distributedacross multiple computing devices, which in turn may also be distributedacross many geographical locations. The patent database 104 may be, butis not limited to, a relational database (e.g., SQL) a non-relationaldatabase, (e.g., NoSQL), or flat file database. For discussion purposes,terms common to relational databases operations are used throughout thisdisclosure.

In various examples, the data analyzer module 102 analyzes data receivedfrom external sources such as the patent data sources 114 before storingdata in the patent database 104. The patent data sources 114 may be anofficial source of patent such as a website or other network accessibledata repository managed by a national patent office. The patent datasources 114 may also be a third-party collector of patent data. The datafrom the patent data sources 114 may be in a raw format. For example,the USPTO offers tagged image file format (TIFF) images and portabledocument format (PDF) files of all public documents for a patent.Foreign patent offices may offer similar data.

In various examples, the patent analytics system 100 request patent datafrom the patent data sources 114 periodically to retrieve file historiesof issued patents or patent applications (collectively referred to aspatents) at the patent data sources 114. The patent analytics system 100may also retrieve overview data provided by the patent data sources 114.The request may use an application programming interface (API) providedby the patent data sources 114 to request the patent data.

After receiving the patent data, the data analyzer module 102 may parsethe raw data or overview data (e.g., using optical characterrecognition, screen scraping, field recognition, etc.) to retrievedetails about each communication to and from the patent Office Action inthe file histories of the patents. FIG. 1 illustrates Office Actiondetails 118 that may be retrieved for an Office Action. As illustrated,the Office Action details 118 may include the serial number of theapplication, the examiner name and what references were cited for eachrejection type. Other details may include the art unit and technologycenter responsible for issuing the Office Action, the supervising patentexaminer (SPE) on the Office Action, and whether an interview wasconducted. The details of each Office Action may be stored in one ormore entries of the patent database 104 and/or the examiner database108. For example, an entry in the patent database 104 may include thefollowing:

-   -   Patent Serial No.    -   Office Action Date    -   Office Action Type [Non-Final, Final, Ex Parte Quayle,        Restriction, etc.]    -   Examiner Name    -   SPE Name    -   Art Unit    -   Tech Center    -   Rejection Type [102, 103, etc.] with cited references by type    -   Number of previous Office Actions to allowance (breakdown by        type [restrictions, final, non-final, RCEs, etc.])    -   Interview conducted? [In-person, telephonic]

In combination with the above described date, the data analyzer module102 may also determine issued patent details 120, including theexaminer's name, assignee at time of issue, length of each independentclaim, references cited during prosecution, number of Office Actions(non-final and final) to issue, and whether or not an appeal was madeduring prosecution. Some of the issued patent details 120 may come fromthe content of the patent itself—as retrieved from the patent datasources 114. These details may be stored in one or more entries of thepatent database 104 and examiner database 108.

The data analyzer module 102 may also analyze the success rate ofarguments made in responses to an Office Action. A successful argumentmay be determined by looking at the art cited against a claim before andafter an argument or by textual analysis of a response to argumentssection of a subsequent Office Action. The data analyzer module 102 mayalso determine the success rate of cited case law by an Applicant in aresponse. For example, the outcome of citing a particular case inresponse to a § 101 rejection may be stored in a database. Over time,this may allow a user to see what cases are most likely to overcome §101 rejections (or 112, 102, 103, etc.) A more granular approach mayalso be used. For example, a user of the application may be able to lookat an individual examiner/art unit/tech center and see what case law hasthe best chance of success.

A user may use his/her computing device to access the data stored at thepatent analytics system 100. In an example, the data is presented viaone or more user interfaces served to the computing device 116. AlthoughFIG. 2 (as well as FIGS. 3 and 4 ) are illustrated as user interfaceswith defined sections, other types of customizable interfaces may beused. For example, a pivot table may be presented to allow a user todetermine their own metrics using the data stored in the patent database104 and the examiner database 108. The data may also be normalized usingfactors set by the user. For example, a user may be able to look at aspecific reference and see how often it is cited by art unit normalizedaccording to the number of filings for the art unit (as compared toother art units). In another example, a user may be able to look at anallowance rate/rejection rate of an examiner normalized to the length ofthe independent claim.

The pivot table (or other interface) may be use to track the performanceof an examiner over time. For example, the allowance rate of theexaminer may be examined when the examiner is a junior examiner, primaryexaminer, and supervisory examiner. If the examiner also becomes atechnology center director the allowance rate of the technology centermay tracked as well. The pivot table may also be used to see how theallowance rate of an examiner changes when the examiner changes artunits.

One example user interface is illustrated in FIG. 2 as examiner overview200. The top of the examiner overview 200 notes the examiner's name aswell as the art unit of the examiner. The information presented inexaminer overview 200 may be gathered (e.g., retrieved or calculated)based on information stored in one or more databases associated with theexaminer as discussed in more detail below. The examiner overview 200may be presented in response to the patent analytics system 100receiving a search query (e.g., with an examiner's name) from thecomputing device 116 via a web page served from the web server 106. Theoverview may also have a date function to see the data for a specificdate range (e.g., the past year). There may also be an option to comparetwo date ranges to see how the data changes over time (e.g., does aparticular assignee get more allowances over time?) Similar overviewsmay be presented for art units and tech centers.

The examiner overview 200 illustrates a variety of sections: atechnology experience section 202; an allowance rate section 204; and areference section 210. The locations of these sections within theinterface, the labels, and the data contained therein are examples—otherlocations may be used. Similarly, more or less data may be included inthe examiner overview 200.

In an example, the experience section 202 includes an overview of theexperience of the examiner as it relates to the examiner's art unit. Theexperience section 202 may include educational credentials and thelength of any relevant work experience. As illustrated, the experiencesection 202 may include a score. The score may be based on a variety offactors depending on the preferences of a user. The formula may useweights for each factor and/or straight values. An example scoringformula may be based on the following components:

-   -   Education component        -   4 pts for having an undergraduate degree relevant to art            unit        -   4 pts for having a graduate degree relevant to art unit    -   USPTO component        -   1 pt. for each year in the art unit    -   Work component        -   1 pt. for each year in relevant job to art unit

An example weighting of the individual components may be:

[(0.4)(Education Component)+(0.2)(USPTO Component)+(0.4)(WorkComponent)]

There may be a default formula to calculate the examiner's technologyexperience score, but it may be modified by a user. For example, a userinterface element may be included in the experience section 202 which,when activated (e.g., clicked), by a user displays the factors that gointo the experience score. A user may select or deselect the factors,change the values given for each factors, and modify the weights for thefactors. The changes may be transmitted to the patent analytics system100 to recalculate the experience score. The examiner overview 200 maybe updated as well in display the recalculated experience score.

The experience section 202 may also include metrics on how theexaminer's credentials compare to other examiners in the art unit andtech center (e.g., an art unit experience comparison score and atechnology center experience comparison score). For example, theexperience section 202 indicates that Examiner Doe has a higherexperience score than 65% of other examiner's in the art unit 0001 andhigher than 70% of examiner's in the examiner's technology center.

In various examples, an experience score module (not shown) of thepatent analytics system 100 calculates the experience. The experiencescore module may retrieve the values for the factors of the experiencescore from one or more databases of the patent analytics system 100. Forexample, the data analyzer module 102 education and work details of anexaminer may be retrieved from the experience sources 112. Theexperience sources 112 may be websites, data services, or datastoresthat include education and work details on a variety of people,including examiners.

Example experience sources may include social networks, professionaldatabases, and company websites. In an example, screen scrapingtechniques are used to retrieve education/work details of a person whenan API is not available at an experience source. The retrievededucation/work details may include, but are not limited to, degreesobtained or in process of being obtained from an educationalinstitution, names of the educational institutions, and names ofbusinesses worked at and starting/ending dates of the same. In variousexamples, the details gathered by the data analyzer module 102 arestored in the examiner database 108. The data analyzer module 102 mayperiodically check the experience sources 112 to retrieve updateddetails.

Using the default experience score formula—or user specified formula—theexperience score module may calculate the experience score for theexaminer. The calculated score may be stored in the examiner database108. The web server 106 may retrieve the calculate score from theexaminer database 108 and include it in experience section 202. In anexample, the experience score is calculated upon request by the user(e.g., the score is not retrieved from a database). In an example, auser requests—via a user interface element—that the score be updated.Accordingly, the data analyzer module 102 may retrieve thework/education detail and the experience score module may calculate theupdated score.

In various examples, the allowance rate section 204 includes allowancevisualization 206 and allowance visualization options 208. The allowancerate section 204 display various allowance metrics for a given examiner.Similar metrics/visualizations may be used to present allowance ratesfor an art unit or tech center. As illustrated, allowance ratevisualizations options may include an overall allowance rate, anallowance rate by assignee, allowance rate by priority date, allowancerate by examiner's time at the USPTO, and allowance rate by time ofyear. Other allowance rate visualization options may also be displayedwithout departing from the scope of this disclosure. The allowancevisualization 206 may be updated in response to a user selecting anallowance rate visualization option.

In an example, the allowance rate by time of year is when in the year anexaminer/art unit/tech center is most likely to allow a case. Dependingon the preferences of a user, the allowance rate may be calculated foreach day/week/month of the year. In an example, the patent analyticssystem 100 may correlate the allowance rate with quotas given to theexaminers. For example, the patent analytics system 100 may compare anexaminer's likelihood to allow cases near the quarter or year-end. Ascore may be given to each examiner based on this comparison. Forexample, a ‘0’ score may mean that allowances are evenly distributedeach week of the year (with an option to normalize given current caseload). A score of ‘1’ may mean that all cases are allowed in the lastmonth of a quarter and a score of ‘−1’ may mean all cases are allowed inthe first month of a quarter—with values also possible in between.Scores may also be calculated for the year. Similar scores may becalculated for art units/tech centers.

In various examples, the reference section 210 displays the most commonreferences cited by the examiner by rejection type. The referencesection 210 also displays options 212 and 214 that, when activated by auser, may retrieve the most common cited references for an art unit/techcenter or additional commonly cited references for the examiner,respectively. If no user of the patent analytics system 100 hasrequested this information before, the patent analytics system 100 maydetermine the most common references by querying the patent database 104using the examiner's name as an input. In an example, after determiningthe most common references (e.g., reference metrics), the examinerdatabase 108 may be updated using this information. A similarly analysismay be performed using the tech unit/art unit/class as an input to thepatent database 104 to determine the most common references for a techunit/art unit/class.

In an example, the patent analytics system 100 provides notificationservices with respect to abandoned patents and expired patents. FIG. 3illustrates an example of abandoned cases notifier interface 300,according to an example embodiment. In an example, the web server 106serves the abandoned cases notifier interface 300 that is rendered on adisplay device of the computing device 116. The abandoned cases notifierinterface 300 includes matter type selections 302, foreign matter option304, and time-period options 306, and address input box 308. A user mayselect which types of matters to be notified of using matter typeselections 302. For example, a user may select a class of patents,certain assignees, or by priority/filing/issue date. The options may beconjunctive or disjunctive.

The user may also choose to have the notification include a list offoreign family matters—regardless of the foreign matter's status. Oftenwhen a U.S. patent goes abandoned, the foreign cases are also left tolapse. Thus, the notified party may become aware of likely abandonedforeign patents.

A user may also be notified of patent applications that might goabandoned in the near future. For example, the patent analytics system100 may send notifications of applications that are nearing a finaldeadline. A user may select a time period using time-period options 306.The user may also enter in one or more e-mail addresses/phone numbers toreceive the notification. Thus, a user may create a notification toreceive a listing of all cases by a specific assignee with a specificclassification that are within two weeks of going abandoned.

FIG. 4 illustrates an example expired cases notifier interface 400,according to an example embodiment. In an example, the web server 106serves the expired cases notifier interface 400 that is rendered on adisplay device of the computing device 116. The expired cases notifierinterface 400 includes matter type selections 402, citation limit option404, and address input box 406. A user may select which types of mattersto be notified of using matter type selections 402. For example, a usermay select a class of patents, certain assignees, or bypriority/filing/issue date. The options may be conjunctive ordisjunctive. An expired matter may be a matter in which an annuity feehas not been paid or a patent with an expired term. In an example, auser may limit the number of matters according to the importance of thematter. For example, a user may choose a forward citation limit usingcitation limit option 404. If an expired patent has less than the limitit will not be included in the notification, in an example.

The patent analytics system 100 may provide a user interface to manage auser's notifications. Thus, a user may receive a listing of allcurrently enabled notifications. The list may also include options todisable or delete the notifications. The patent analytics system 100 mayalso include options to change the frequency of notifications (e.g., aweekly e-mail including all notifications).

Example Computer System

Embodiments described herein may be implemented in one or a combinationof hardware, firmware, and software. Embodiments may also be implementedas instructions stored on a machine-readable storage device, which maybe read and executed by at least one processor to perform the operationsdescribed herein. A machine-readable storage device may include anynon-transitory mechanism for storing information in a form readable by amachine (e.g., a computer). For example, a machine-readable storagedevice may include read-only memory (ROM), random-access memory (RAM),magnetic disk storage media, optical storage media, flash-memorydevices, and other storage devices and media.

Examples, as described herein, may include, or may operate on, logic ora number of components, modules, or mechanisms. Modules may be hardware,software, or firmware communicatively coupled to one or more processorsin order to carry out the operations described herein. Modules mayhardware modules, and as such modules may be considered tangibleentities capable of performing specified operations and may beconfigured or arranged in a certain manner. In an example, circuits maybe arranged (e.g., internally or with respect to external entities suchas other circuits) in a specified manner as a module. In an example, thewhole or part of one or more computer systems (e.g., a standalone,client or server computer system) or one or more hardware processors maybe configured by firmware or software (e.g., instructions, anapplication portion, or an application) as a module that operates toperform specified operations. In an example, the software may reside ona machine-readable medium. In an example, the software, when executed bythe underlying hardware of the module, causes the hardware to performthe specified operations. Accordingly, the term hardware module isunderstood to encompass a tangible entity, be that an entity that isphysically constructed, specifically configured (e.g., hardwired), ortemporarily (e.g., transitorily) configured (e.g., programmed) tooperate in a specified manner or to perform part or all of any operationdescribed herein. Considering examples in which modules are temporarilyconfigured, each of the modules need not be instantiated at any onemoment in time. For example, where the modules comprise ageneral-purpose hardware processor configured using software; thegeneral-purpose hardware processor may be configured as respectivedifferent modules at different times. Software may accordingly configurea hardware processor, for example, to constitute a particular module atone instance of time and to constitute a different module at a differentinstance of time. Modules may also be software or firmware modules,which operate to perform the methodologies described herein.

FIG. 5 is a block diagram illustrating a machine in the example form ofa computer system 500, within which a set or sequence of instructionsmay be executed to cause the machine to perform any one of themethodologies discussed herein, according to an example embodiment. Inalternative embodiments, the machine operates as a standalone device ormay be connected (e.g., networked) to other machines. In a networkeddeployment, the machine may operate in the capacity of either a serveror a client machine in server-client network environments, or it may actas a peer machine in peer-to-peer (or distributed) network environments.The machine may be an onboard vehicle system, wearable device, personalcomputer (PC), a tablet PC, a hybrid tablet, a personal digitalassistant (PDA), a mobile telephone, or any machine capable of executinginstructions (sequential or otherwise) that specify actions to be takenby that machine. Further, while only a single machine is illustrated,the term “machine” shall also be taken to include any collection ofmachines that individually or jointly execute a set (or multiple sets)of instructions to perform any one or more of the methodologiesdiscussed herein. Similarly, the term “processor-based system” shall betaken to include any set of one or more machines that are controlled byor operated by a processor (e.g., a computer) to individually or jointlyexecute instructions to perform any one or more of the methodologiesdiscussed herein.

Example computer system 500 includes at least one processor 502 (e.g., acentral processing unit (CPU), a graphics processing unit (GPU) or both,processor cores, compute nodes, etc.), a main memory 504 and a staticmemory 506, which communicate with each other via a link 508 (e.g.,bus). The computer system 500 may further include a video display unit510, an alphanumeric input device 512 (e.g., a keyboard), and a userinterface (UI) navigation device 514 (e.g., a mouse). In one embodiment,the video display unit 510, input device 512 and UI navigation device514 are incorporated into a touch screen display. The computer system500 may additionally include a storage device 516 (e.g., a drive unit),a signal generation device 518 (e.g., a speaker), a network interfacedevice 520, and one or more sensors (not shown), such as a globalpositioning system (GPS) sensor, compass, accelerometer, or othersensor.

The storage device 516 includes a machine-readable medium 522 on whichis stored one or more sets of data structures and instructions 524(e.g., software) embodying or utilized by any one or more of themethodologies or functions described herein. The instructions 524 mayalso reside, completely or at least partially, within the main memory504, static memory 506, and/or within the processor 502 during executionthereof by the computer system 500, with the main memory 504, staticmemory 506, and the processor 502 also constituting machine-readablemedia.

While the machine-readable medium 522 is illustrated in an exampleembodiment to be a single medium, the term “machine-readable medium” mayinclude a single medium or multiple media (e.g., a centralized ordistributed database, and/or associated caches and servers) that storethe one or more instructions 524. The term “machine-readable medium”shall also be taken to include any tangible medium that is capable ofstoring, encoding or carrying instructions for execution by the machineand that cause the machine to perform any one or more of themethodologies of the present disclosure or that is capable of storing,encoding or carrying data structures utilized by or associated with suchinstructions. The term “machine-readable medium” shall accordingly betaken to include, but not be limited to, solid-state memories, andoptical and magnetic media. Specific examples of machine-readable mediainclude non-volatile memory, including but not limited to, by way ofexample, semiconductor memory devices (e.g., electrically programmableread-only memory (EPROM), electrically erasable programmable read-onlymemory (EEPROM)) and flash memory devices; magnetic disks such asinternal hard disks and removable disks; magneto-optical disks; andCD-ROM and DVD-ROM disks.

The instructions 524 may further be transmitted or received over acommunications network 526 using a transmission medium via the networkinterface device 520 utilizing any one of a number of well-knowntransfer protocols (e.g., HTTP). Examples of communication networksinclude a local area network (LAN), a wide area network (WAN), theInternet, mobile telephone networks, plain old telephone (POTS)networks, and wireless data networks (e.g., Wi-Fi, 3G, and 4G LTE/LTE-Aor WiMAX networks). The term “transmission medium” shall be taken toinclude any intangible medium that is capable of storing, encoding, orcarrying instructions for execution by the machine, and includes digitalor analog communications signals or other intangible medium tofacilitate communication of such software.

The above detailed description includes references to the accompanyingdrawings, which form a part of the detailed description. The drawingsshow, by way of illustration, specific embodiments that may bepracticed. These embodiments are also referred to herein as “examples.”Such examples may include elements in addition to those shown ordescribed. However, also contemplated are examples that include theelements shown or described. Moreover, also contemplate are examplesusing any combination or permutation of those elements shown ordescribed (or one or more aspects thereof), either with respect to aparticular example (or one or more aspects thereof), or with respect toother examples (or one or more aspects thereof) shown or describedherein.

1. (canceled)
 2. A system for identifying commonly cited references,comprising: at least one processor; and memory including instructionsthat, when executed by the at least one processor, cause the at leastone processor to perform operations to: generate an applicationprogramming interface (API) call for obtaining examiner data for anexaminer in a patent office from a set of experience sources; submit theAPI call to the set of experience sources; parse the examiner data,retrieved in response to the API call, to store patent prosecution datafor the examiner in at least one database; generate a user interface fordisplay on a computing device as a pivot table; receive, via the userinterface, a search query, the search query identifying the examiner inthe patent office; query the at least one database with the searchquery; in response to the query, receive results from the at least onedatabase, the results including references cited by the examiner forpatent rejections and allowance metrics that include case law referencescited in office action responses against patent rejections of theexaminer; display on the user interface in the pivot table, a name forthe examiner and a. reference section, wherein the reference sectionincludes reference metrics, determination of the reference metricscomprising operations to: calculate for each respective reference of thereferences cited, a total number of instances where a respectivereference was cited by the examiner; select references for a list ofmost common references cited by the examiner in rejections based on atotal calculated for each respective reference of the references cited;transmit an update command to the at least one database for an entry ofthe examiner with a list of common references cited by the examiner inrejections; calculate for each respective case law reference of the caselaw references cited in office action responses against patentrejections, a total number of instances where a respective case lawreference was cited in office action responses against patent rejectionsfor patent applications which were allowed; and select case lawreferences for a list of most common case law references cited in officeaction responses against patent rejections for allowed patentapplications based on the total calculated for each respective case lawreference of the case law references cited; and. display in thereference section of the user interface, the list of most commonreferences cited by the examiner in rejections based on the totalcalculated for each respective reference of the references cited.
 3. Thesystem of claim 2, wherein the reference section indicates a percentageof cases a reference of the list of most common references is used in.4. The system of claim 2, wherein the pivot table includes an experiencesection and the memory further comprising instructions that, whenexecuted by the at least one processor, cause the at least one processorto perform operations to: parse the examiner data, retrieved in responseto the API call to identify experience metrics based on educationalexperience, patent office experience, and work experience of theexaminer; generate a technology experience score for the examiner basedon the experience metrics, wherein the technology experience score isgenerated by applying an educational experience score, a patent officeexperience score, and a work experience score to a technology scoringformula, wherein the instructions to generate the technology experiencescore comprises instructions that case the at least one processor toperform operations to: calculate the educational experience score as asum of numerical values assigned to educational degrees in theeducational experience based on determined relevance to a patent artunit of the patent office; calculate the patent office experience scoreas a sum of numerical values assigned to years the examiner has workedin the patent art unit as determined from the patent office experience;and calculate the work experience score as a sum of numerical valuesassigned to years worked in jobs determined relevance to the patent artunit in the work experience; and display the technology experience scorein the experience section of the pivot table.
 5. The system of claim 4,the memory further comprising instructions that, when executed by the atleast one processor, cause the at least one processor to performoperations to: compare the technology experience score for the examinerto technology experience scores for other examiners in an art unit ofthe examiner to determine an art unit experience comparison score,wherein the experience section includes the art unit experiencecomparison score.
 6. The system of claim 2, wherein the pivot tableincludes an allowance section and the memory further comprisinginstructions that, when executed by the at least one processor, causethe at least one processor to perform operations to: display in theallowance section: the allowance metrics; allowance visualizationoptions; an allowance visualization based on the allowance metrics and aselected visualization option of the allowance visualization options;and the list of most common case law references cited in office actionresponses against patent rejections for allowed patent applicationsbased on the total calculated for each respective case law reference ofthe case law references cited.
 7. The system of claim 6, wherein theallowance visualization options include an option to display anallowance rate for the examiner by assignee and wherein the allowancevisualization is updated based on selection of the option.
 8. The systemof claim 6, wherein the allowance visualization options include anoption to display an allowance rate for the examiner by time of year andwherein the allowance visualization is updated based on selection of theoption.
 9. At least one non-transitory machine-readable medium forinstructions for identifying commonly cited references that, whenexecuted by at least one processor, cause the at least one processor toperform operations to: generate an application programming interface(API) call for obtaining examiner data for an examiner in a patentoffice from a set of experience sources; submit the API call to the setof experience sources; parse the examiner data, retrieved in response tothe API call, to stoic patent prosecution data for the examiner in atleast one database; generate a user interface for display on a computingdevice as a pivot table; receive, via the user interface, a searchquery, the search query identifying the examiner in the patent office;query the at least one database with the search query; in response tothe query, receive results from the at least one database, the resultsincluding references cited by the examiner for patent rejections andallowance metrics that include case law references cited in officeaction responses against patent rejections of the examiner; display onthe user interface in the pivot table, a name for the examiner and areference section, wherein the reference section includes referencemetrics, determination of the reference metrics comprising operationsto: calculate for each respective reference of the references cited, atotal number of instances where a respective reference was cited by theexaminer; select references for a list of most common references citedby the examiner in rejections based on a total calculated for eachrespective reference of the references cited; transmit an update commandto the at least one database for an entry of the examiner with a list ofcommon references cited by the examiner in rejections; calculate foreach respective case law reference of the case law references cited inoffice action responses against patent rejections, a total number ofinstances where a respective case law reference was cited in officeaction responses against patent rejections for patent applications whichwere allowed; and select case law references for a list of most commoncase law references cited in office action responses against patentrejections for allowed patent applications based on the total calculatedfor each respective case law reference of the case law references cited;and display in the reference section of the user interface, the list ofmost common references cited by the examiner in rejections based on thetotal calculated for each respective reference of the references cited.10. The at least one non-transitory machine-readable medium of claim 9,Wherein the reference section indicates a percentage of cases areference of the list of most common references is used in.
 11. The atleast one non-transitory machine-readable medium of claim 9, wherein thepivot table includes an experience section and further comprisinginstructions that, when executed by the at least one processor, causethe at least one processor to perform operations to: parse the examinerdata, retrieved in response to the API call to identify experiencemetrics based on educational experience, patent office experience, andwork experience of the examiner; generate a technology experience scorefor the examiner based on the experience metrics, wherein the technologyexperience score is generated by applying an educational experiencescore, a patent office experience score, and a work experience score toa technology scoring formula, wherein the instructions to generate thetechnology experience score comprises instructions that case the atleast one processor to perform operations to: calculate the educationalexperience score as a sum of numerical values assigned to educationaldegrees in the educational experience based on determined relevance to apatent art unit of the patent office; calculate the patent officeexperience score as a sum of numerical values assigned to years theexaminer has worked in the patent art unit as determined from the patentoffice experience; and calculate the work experience score as a sum ofnumerical values assigned to yearsworked in jobs determined relevance tothe patent art unit in the work experience; and display the technologyexperience score in the experience section of the pivot table.
 12. Theat least one non-transitory machine-readable medium of claim 11, furthercomprising instructions that, when executed by the at least oneprocessor, cause the at least one processor to perform operations to:compare the technology experience score for the examiner to technologyexperience scores for other examiners in an art unit of the examiner todetermine an art unit experience comparison score, wherein theexperience section includes the art unit experience comparison score.13. The at least one non-transitory machine-readable medium of claim 9,Wherein the pivot table includes an allowance section and furthercomprising instructions that, when executed by the at least oneprocessor, cause the at least one processor to perform operations to:display in the allowance section: the allowance metrics; allowancevisualization options; an allowance visualization based on the allowancemetrics and a selected visualization option of the allowancevisualization options; and the list of most common case law referencescited in office action responses against patent rejections for allowedpatent applications based on the total calculated for each respectivecase law reference of the case law references cited.
 14. The at leastone non-transitory machine-readable medium of claim 13, wherein theallowance visualization options include an option to display anallowance rate for the examiner by assignee and wherein the allowancevisualization is updated based on selection of the option.
 15. The atleast one non-transitory machine-readable medium of claim 13, whereinthe allowance visualization options include an option to display anallowance rate for the examiner by time of year and wherein theallowance visualization is updated based on selection of the option. 16.A method for identifying commonly cited references, comprising:generating an application programming interface (API) call for obtainingexaminer data. for an examiner in a patent office from a set ofexperience sources; submitting the API call to the set of experiencesources; parsing the examiner data, retrieved in response to the APIcall, to store patent prosecution data for the examiner in at least onedatabase; generating a user interface for display on a computing deviceas a pivot table; receiving, via the user interface, a search query, thesearch query identifying the examiner in the patent office; querying theat least one database with the search query; in response to thequerying, receiving results from the at least one database, the resultsincluding references cited by the examiner for patent rejections andallowance metrics that include case law references cited in officeaction responses against patent rejections of the examiner; displayingon the user interface in the pivot table, a name for the examiner and areference section, wherein the reference section includes referencemetrics, wherein the reference metrics are determined by: calculatingfor each respective reference of the references cited, a total number ofinstances where a respective reference was cited by the examiner;selecting references for a list of most common references cited by theexaminer in rejections based on a total calculated for each respectivereference of the references cited; transmitting an update command to theat least one database for an entry of the examiner with a list of commonreferences cited by the examiner in rejections; calculating for eachrespective case law reference of the case law references cited in officeaction responses against patent rejections, a total number of instanceswhere a respective case law reference was cited in office actionresponses against patent rejections for patent applications which wereallowed; and selecting case law references for a list of most commoncase law references cited in office action responses against patentrejections for allowed patent applications based on the total calculatedfor each respective case law reference of the case law references cited;and displaying in the reference section of the user interface, the listof most common references cited by the examiner in rejections based onthe total calculated for each respective reference of the referencescited.
 17. The method of claim 16, wherein the reference sectionindicates a percentage of cases a reference of the list of most commonreferences is used in.
 18. The method of claim 16, wherein the pivottable includes an experience section and further comprising: parsing theexaminer data, retrieved in response to the API call to identifyexperience metrics based on educational experience, patent officeexperience, and work experience of the examiner; generating a technologyexperience score for the examiner based on the experience metrics,wherein the technology experience score is generated by applying aneducational experience score, a patent office experience score, and awork experience score to a technology scoring formula, whereingenerating the technology experience score comprises: calculating theeducational experience score as a sum of numerical values assigned toeducational degrees in the educational experience based on determinedrelevance to a patent art unit of the patent office; calculating thepatent office experience score as a sum of numerical values assigned toyears the examiner has worked in the patent art unit as determined fromthe patent office experience; and calculating the work experience scoreas a sum of numerical values assigned to years worked in jobs determinedrelevance to the patent art unit in the work experience; and displayingthe technology experience score in the experience section of the pivottable.
 19. The method of claim 18, further comprising: comparing thetechnology experience score for the examiner to technology experiencescores for other examiners in an art unit of the examiner to determinean art unit experience comparison score, wherein the experience sectionincludes the art unit experience comparison score.
 20. The method ofclaim 16, wherein the pivot table includes an allowance section andfurther comprising: displaying in the allowance section: the allowancemetrics; allowance visualization options; an allowance visualizationbased on the allowance metrics and a selected visualization option ofthe allowance visualization options; and the list of most common caselaw references cited in office action responses against patentrejections for allowed patent applications based on the total calculatedfor each respective case law reference of the case law references cited.21. The method of claim 20, wherein the allowance visualization optionsinclude an option to display an allowance rate for the examiner byassignee and wherein the allowance visualization is updated based onselection of the option.